{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NVIDIA Inference Server MNIST Example\n",
    "\n",
    "This example shows how you can combine Seldon with the NVIDIA Inference Server. We will use a Seldon TensorRT proxy model image that will forward Seldon internal microservice prediction calls out to an external TensorRT Inference Server.\n",
    "\n",
    "The example will use the MNIST digit classification task with a pre-trained CAFFE2 model.\n",
    "\n",
    "A Seldon transformer will transform the inputs before sending to the Proxy which will forward the request to the Nvidia Inference Server.\n",
    "\n",
    "This example will:\n",
    "\n",
    " * Show the packaging of the components using S2I and a step by step local testing of these via Docker\n",
    " * Show running the example in Seldon Core on GCP with an embedded Nvidia Inference Server\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import requests\n",
    "from random import randint,random\n",
    "import json\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "import sys\n",
    "sys.path.append(\"../../../notebooks\")\n",
    "from visualizer import get_graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gen_image(arr):\n",
    "    two_d = (np.reshape(arr, (28, 28)) * 255).astype(np.uint8)\n",
    "    plt.imshow(two_d,cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "    return plt\n",
    "\n",
    "def download_mnist():\n",
    "    return input_data.read_data_sets(\"MNIST_data/\", one_hot = True)\n",
    "\n",
    "def rest_predict_request(endpoint,data):\n",
    "    request = {\"data\":{\"ndarray\":data.tolist()}}\n",
    "    response = requests.post(\n",
    "                \"http://\"+endpoint+\"/predict\",\n",
    "                data={\"json\":json.dumps(request),\"isDefault\":True})\n",
    "    return response.json()   \n",
    "\n",
    "def rest_transform_input_request(endpoint,data):\n",
    "    request = {\"data\":{\"ndarray\":data.tolist()}}\n",
    "    response = requests.post(\n",
    "                \"http://\"+endpoint+\"/transform-input\",\n",
    "                data={\"json\":json.dumps(request),\"isDefault\":True})\n",
    "    return response.json()   \n",
    "\n",
    "def rest_transform_output_request(endpoint,data):\n",
    "    request = {\"data\":{\"ndarray\":data.tolist()}}\n",
    "    response = requests.post(\n",
    "                \"http://\"+endpoint+\"/transform-output\",\n",
    "                data={\"json\":json.dumps(request),\"isDefault\":True})\n",
    "    return response.json()   \n",
    "\n",
    "def rest_request_ambassador(deploymentName,endpoint=\"localhost:8003\",arr=None):\n",
    "    payload = {\"data\":{\"names\":[\"a\",\"b\"],\"tensor\":{\"shape\":[1,784],\"values\":arr.tolist()}}}\n",
    "    response = requests.post(\n",
    "                \"http://\"+endpoint+\"/seldon/seldon/\"+deploymentName+\"/api/v0.1/predictions\",\n",
    "                json=payload)\n",
    "    print(response.status_code)\n",
    "    print(response.text)\n",
    "\n",
    "\n",
    "def gen_mnist_data(mnist):\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(1)\n",
    "    chosen=0\n",
    "    gen_image(batch_xs[chosen]).show()\n",
    "    data = batch_xs[chosen].reshape((1,784))\n",
    "    return data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-24-8029beac92e7>:7: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use urllib or similar directly.\n",
      "Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n",
      "Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From /home/clive/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n"
     ]
    }
   ],
   "source": [
    "mnist = download_mnist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create an Nvidia Model Repository"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fetch pretrained MNIST model ready for serving and place in model repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p tensorrt_mnist/1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-04-16 18:35:23--  http://seldon-public.s3.amazonaws.com/nvidia-mnist-model/model.plan\n",
      "Resolving seldon-public.s3.amazonaws.com (seldon-public.s3.amazonaws.com)... 52.218.49.89\n",
      "Connecting to seldon-public.s3.amazonaws.com (seldon-public.s3.amazonaws.com)|52.218.49.89|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1731864 (1.7M) [binary/octet-stream]\n",
      "Saving to: ‘tensorrt_mnist/1/model.plan’\n",
      "\n",
      "tensorrt_mnist/1/mo 100%[===================>]   1.65M   846KB/s    in 2.0s    \n",
      "\n",
      "2019-04-16 18:35:25 (846 KB/s) - ‘tensorrt_mnist/1/model.plan’ saved [1731864/1731864]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O tensorrt_mnist/1/model.plan http://seldon-public.s3.amazonaws.com/nvidia-mnist-model/model.plan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run your Mvidia Inference Server you will need to upload to a google bucket the model repository in mnsit_tensorrt_model. Follow the steps below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: MODEL_REPOSITORY_BUCKET=gs://seldon-inference-server-model-store\n"
     ]
    }
   ],
   "source": [
    "# CHANGE THIS TO YOUR OWN CHOSEN GOOGLE BUCKET NAME\n",
    "%env MODEL_REPOSITORY_BUCKET=gs://seldon-inference-server-model-store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating gs://seldon-inference-server-model-store/...\n",
      "ServiceException: 409 Bucket seldon-inference-server-model-store already exists.\n"
     ]
    }
   ],
   "source": [
    "!gsutil mb ${MODEL_REPOSITORY_BUCKET}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying file://tensorrt_mnist/config.pbtxt [Content-Type=application/octet-stream]...\n",
      "Copying file://tensorrt_mnist/mnist_labels.txt [Content-Type=text/plain]...     \n",
      "Copying file://tensorrt_mnist/1/model.plan [Content-Type=application/octet-stream]...\n",
      "- [3 files][  1.7 MiB/  1.7 MiB]                                                \n",
      "Operation completed over 3 objects/1.7 MiB.                                      \n"
     ]
    }
   ],
   "source": [
    "!gsutil cp -r tensorrt_mnist ${MODEL_REPOSITORY_BUCKET}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create your Nvidia Inference Server\n",
    "\n",
    "For example:\n",
    "\n",
    " * Follow the steps in the [Kubeflow guide](https://github.com/kubeflow/kubeflow/tree/master/kubeflow/nvidia-inference-server) to create your Nvidia Inference Server \n",
    "   * You will need to use the Google Bucket location"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Package and run a Transformer and Nvidia Proxy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use a Seldon Transformer to remove the training-set mean values from the input features and rehsape the output as the prediction comes back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mimport\u001b[39;49;00m \u001b[04m\u001b[36mnumpy\u001b[39;49;00m \u001b[34mas\u001b[39;49;00m \u001b[04m\u001b[36mnp\u001b[39;49;00m\r\n",
      "\r\n",
      "MEANS=np.array([\u001b[34m255.0\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m235\u001b[39;49;00m,\u001b[34m228\u001b[39;49;00m,\u001b[34m220\u001b[39;49;00m,\u001b[34m213\u001b[39;49;00m,\u001b[34m210\u001b[39;49;00m,\u001b[34m211\u001b[39;49;00m,\u001b[34m216\u001b[39;49;00m,\u001b[34m224\u001b[39;49;00m,\u001b[34m232\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m234\u001b[39;49;00m,\u001b[34m223\u001b[39;49;00m,\u001b[34m211\u001b[39;49;00m,\u001b[34m196\u001b[39;49;00m,\u001b[34m181\u001b[39;49;00m,\u001b[34m170\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m166\u001b[39;49;00m,\u001b[34m175\u001b[39;49;00m,\u001b[34m189\u001b[39;49;00m,\u001b[34m205\u001b[39;49;00m,\u001b[34m221\u001b[39;49;00m,\u001b[34m233\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m241\u001b[39;49;00m,\u001b[34m231\u001b[39;49;00m,\u001b[34m217\u001b[39;49;00m,\u001b[34m202\u001b[39;49;00m,\u001b[34m184\u001b[39;49;00m,\u001b[34m166\u001b[39;49;00m,\u001b[34m149\u001b[39;49;00m,\u001b[34m136\u001b[39;49;00m,\u001b[34m131\u001b[39;49;00m,\u001b[34m134\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m159\u001b[39;49;00m,\u001b[34m180\u001b[39;49;00m,\u001b[34m201\u001b[39;49;00m,\u001b[34m220\u001b[39;49;00m,\u001b[34m234\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m233\u001b[39;49;00m,\u001b[34m219\u001b[39;49;00m,\u001b[34m201\u001b[39;49;00m,\u001b[34m181\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m130\u001b[39;49;00m,\u001b[34m122\u001b[39;49;00m,\u001b[34m120\u001b[39;49;00m,\u001b[34m122\u001b[39;49;00m,\u001b[34m129\u001b[39;49;00m,\u001b[34m141\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m185\u001b[39;49;00m,\u001b[34m208\u001b[39;49;00m,\u001b[34m227\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m238\u001b[39;49;00m,\u001b[34m226\u001b[39;49;00m,\u001b[34m208\u001b[39;49;00m,\u001b[34m187\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m146\u001b[39;49;00m,\u001b[34m135\u001b[39;49;00m,\u001b[34m131\u001b[39;49;00m,\u001b[34m132\u001b[39;49;00m,\u001b[34m133\u001b[39;49;00m,\u001b[34m132\u001b[39;49;00m,\u001b[34m133\u001b[39;49;00m,\u001b[34m139\u001b[39;49;00m,\u001b[34m154\u001b[39;49;00m,\u001b[34m178\u001b[39;49;00m,\u001b[34m202\u001b[39;49;00m,\u001b[34m223\u001b[39;49;00m,\u001b[34m239\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m236\u001b[39;49;00m,\u001b[34m221\u001b[39;49;00m,\u001b[34m200\u001b[39;49;00m,\u001b[34m177\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m144\u001b[39;49;00m,\u001b[34m144\u001b[39;49;00m,\u001b[34m150\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m151\u001b[39;49;00m,\u001b[34m144\u001b[39;49;00m,\u001b[34m144\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m178\u001b[39;49;00m,\u001b[34m202\u001b[39;49;00m,\u001b[34m224\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m235\u001b[39;49;00m,\u001b[34m218\u001b[39;49;00m,\u001b[34m195\u001b[39;49;00m,\u001b[34m172\u001b[39;49;00m,\u001b[34m155\u001b[39;49;00m,\u001b[34m152\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m172\u001b[39;49;00m,\u001b[34m176\u001b[39;49;00m,\u001b[34m170\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m150\u001b[39;49;00m,\u001b[34m149\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m183\u001b[39;49;00m,\u001b[34m207\u001b[39;49;00m,\u001b[34m227\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m234\u001b[39;49;00m,\u001b[34m215\u001b[39;49;00m,\u001b[34m191\u001b[39;49;00m,\u001b[34m168\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m160\u001b[39;49;00m,\u001b[34m173\u001b[39;49;00m,\u001b[34m182\u001b[39;49;00m,\u001b[34m179\u001b[39;49;00m,\u001b[34m169\u001b[39;49;00m,\u001b[34m157\u001b[39;49;00m,\u001b[34m147\u001b[39;49;00m,\u001b[34m149\u001b[39;49;00m,\u001b[34m166\u001b[39;49;00m,\u001b[34m190\u001b[39;49;00m,\u001b[34m213\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m233\u001b[39;49;00m,\u001b[34m212\u001b[39;49;00m,\u001b[34m186\u001b[39;49;00m,\u001b[34m165\u001b[39;49;00m,\u001b[34m157\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m175\u001b[39;49;00m,\u001b[34m176\u001b[39;49;00m,\u001b[34m165\u001b[39;49;00m,\u001b[34m153\u001b[39;49;00m,\u001b[34m142\u001b[39;49;00m,\u001b[34m137\u001b[39;49;00m,\u001b[34m147\u001b[39;49;00m,\u001b[34m170\u001b[39;49;00m,\u001b[34m196\u001b[39;49;00m,\u001b[34m217\u001b[39;49;00m,\u001b[34m231\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m207\u001b[39;49;00m,\u001b[34m182\u001b[39;49;00m,\u001b[34m163\u001b[39;49;00m,\u001b[34m158\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m168\u001b[39;49;00m,\u001b[34m158\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m131\u001b[39;49;00m,\u001b[34m125\u001b[39;49;00m,\u001b[34m128\u001b[39;49;00m,\u001b[34m146\u001b[39;49;00m,\u001b[34m174\u001b[39;49;00m,\u001b[34m200\u001b[39;49;00m,\u001b[34m218\u001b[39;49;00m,\u001b[34m231\u001b[39;49;00m,\u001b[34m241\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m227\u001b[39;49;00m,\u001b[34m205\u001b[39;49;00m,\u001b[34m181\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m159\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m157\u001b[39;49;00m,\u001b[34m139\u001b[39;49;00m,\u001b[34m124\u001b[39;49;00m,\u001b[34m115\u001b[39;49;00m,\u001b[34m118\u001b[39;49;00m,\u001b[34m127\u001b[39;49;00m,\u001b[34m148\u001b[39;49;00m,\u001b[34m176\u001b[39;49;00m,\u001b[34m199\u001b[39;49;00m,\u001b[34m216\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m241\u001b[39;49;00m,\u001b[34m224\u001b[39;49;00m,\u001b[34m204\u001b[39;49;00m,\u001b[34m184\u001b[39;49;00m,\u001b[34m169\u001b[39;49;00m,\u001b[34m163\u001b[39;49;00m,\u001b[34m160\u001b[39;49;00m,\u001b[34m150\u001b[39;49;00m,\u001b[34m132\u001b[39;49;00m,\u001b[34m119\u001b[39;49;00m,\u001b[34m116\u001b[39;49;00m,\u001b[34m123\u001b[39;49;00m,\u001b[34m133\u001b[39;49;00m,\u001b[34m153\u001b[39;49;00m,\u001b[34m177\u001b[39;49;00m,\u001b[34m197\u001b[39;49;00m,\u001b[34m214\u001b[39;49;00m,\u001b[34m228\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m239\u001b[39;49;00m,\u001b[34m222\u001b[39;49;00m,\u001b[34m205\u001b[39;49;00m,\u001b[34m189\u001b[39;49;00m,\u001b[34m177\u001b[39;49;00m,\u001b[34m171\u001b[39;49;00m,\u001b[34m166\u001b[39;49;00m,\u001b[34m154\u001b[39;49;00m,\u001b[34m139\u001b[39;49;00m,\u001b[34m129\u001b[39;49;00m,\u001b[34m128\u001b[39;49;00m,\u001b[34m134\u001b[39;49;00m,\u001b[34m144\u001b[39;49;00m,\u001b[34m159\u001b[39;49;00m,\u001b[34m177\u001b[39;49;00m,\u001b[34m195\u001b[39;49;00m,\u001b[34m213\u001b[39;49;00m,\u001b[34m228\u001b[39;49;00m,\u001b[34m241\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m237\u001b[39;49;00m,\u001b[34m222\u001b[39;49;00m,\u001b[34m207\u001b[39;49;00m,\u001b[34m195\u001b[39;49;00m,\u001b[34m186\u001b[39;49;00m,\u001b[34m180\u001b[39;49;00m,\u001b[34m175\u001b[39;49;00m,\u001b[34m166\u001b[39;49;00m,\u001b[34m153\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m140\u001b[39;49;00m,\u001b[34m142\u001b[39;49;00m,\u001b[34m150\u001b[39;49;00m,\u001b[34m162\u001b[39;49;00m,\u001b[34m178\u001b[39;49;00m,\u001b[34m195\u001b[39;49;00m,\u001b[34m214\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m235\u001b[39;49;00m,\u001b[34m220\u001b[39;49;00m,\u001b[34m207\u001b[39;49;00m,\u001b[34m197\u001b[39;49;00m,\u001b[34m189\u001b[39;49;00m,\u001b[34m183\u001b[39;49;00m,\u001b[34m179\u001b[39;49;00m,\u001b[34m172\u001b[39;49;00m,\u001b[34m160\u001b[39;49;00m,\u001b[34m148\u001b[39;49;00m,\u001b[34m142\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m150\u001b[39;49;00m,\u001b[34m161\u001b[39;49;00m,\u001b[34m178\u001b[39;49;00m,\u001b[34m198\u001b[39;49;00m,\u001b[34m217\u001b[39;49;00m,\u001b[34m233\u001b[39;49;00m,\u001b[34m244\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m233\u001b[39;49;00m,\u001b[34m218\u001b[39;49;00m,\u001b[34m204\u001b[39;49;00m,\u001b[34m192\u001b[39;49;00m,\u001b[34m184\u001b[39;49;00m,\u001b[34m177\u001b[39;49;00m,\u001b[34m172\u001b[39;49;00m,\u001b[34m165\u001b[39;49;00m,\u001b[34m153\u001b[39;49;00m,\u001b[34m142\u001b[39;49;00m,\u001b[34m137\u001b[39;49;00m,\u001b[34m139\u001b[39;49;00m,\u001b[34m148\u001b[39;49;00m,\u001b[34m163\u001b[39;49;00m,\u001b[34m183\u001b[39;49;00m,\u001b[34m204\u001b[39;49;00m,\u001b[34m222\u001b[39;49;00m,\u001b[34m236\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m234\u001b[39;49;00m,\u001b[34m218\u001b[39;49;00m,\u001b[34m201\u001b[39;49;00m,\u001b[34m186\u001b[39;49;00m,\u001b[34m174\u001b[39;49;00m,\u001b[34m165\u001b[39;49;00m,\u001b[34m157\u001b[39;49;00m,\u001b[34m148\u001b[39;49;00m,\u001b[34m137\u001b[39;49;00m,\u001b[34m130\u001b[39;49;00m,\u001b[34m129\u001b[39;49;00m,\u001b[34m137\u001b[39;49;00m,\u001b[34m151\u001b[39;49;00m,\u001b[34m171\u001b[39;49;00m,\u001b[34m194\u001b[39;49;00m,\u001b[34m214\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m238\u001b[39;49;00m,\u001b[34m222\u001b[39;49;00m,\u001b[34m203\u001b[39;49;00m,\u001b[34m184\u001b[39;49;00m,\u001b[34m168\u001b[39;49;00m,\u001b[34m154\u001b[39;49;00m,\u001b[34m143\u001b[39;49;00m,\u001b[34m132\u001b[39;49;00m,\u001b[34m124\u001b[39;49;00m,\u001b[34m123\u001b[39;49;00m,\u001b[34m130\u001b[39;49;00m,\u001b[34m145\u001b[39;49;00m,\u001b[34m165\u001b[39;49;00m,\u001b[34m188\u001b[39;49;00m,\u001b[34m209\u001b[39;49;00m,\u001b[34m227\u001b[39;49;00m,\u001b[34m239\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m244\u001b[39;49;00m,\u001b[34m232\u001b[39;49;00m,\u001b[34m214\u001b[39;49;00m,\u001b[34m194\u001b[39;49;00m,\u001b[34m174\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m142\u001b[39;49;00m,\u001b[34m132\u001b[39;49;00m,\u001b[34m130\u001b[39;49;00m,\u001b[34m134\u001b[39;49;00m,\u001b[34m148\u001b[39;49;00m,\u001b[34m167\u001b[39;49;00m,\u001b[34m189\u001b[39;49;00m,\u001b[34m210\u001b[39;49;00m,\u001b[34m226\u001b[39;49;00m,\u001b[34m238\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m231\u001b[39;49;00m,\u001b[34m215\u001b[39;49;00m,\u001b[34m196\u001b[39;49;00m,\u001b[34m178\u001b[39;49;00m,\u001b[34m163\u001b[39;49;00m,\u001b[34m155\u001b[39;49;00m,\u001b[34m156\u001b[39;49;00m,\u001b[34m164\u001b[39;49;00m,\u001b[34m179\u001b[39;49;00m,\u001b[34m197\u001b[39;49;00m,\u001b[34m215\u001b[39;49;00m,\u001b[34m230\u001b[39;49;00m,\u001b[34m240\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m246\u001b[39;49;00m,\u001b[34m238\u001b[39;49;00m,\u001b[34m228\u001b[39;49;00m,\u001b[34m217\u001b[39;49;00m,\u001b[34m208\u001b[39;49;00m,\u001b[34m203\u001b[39;49;00m,\u001b[34m204\u001b[39;49;00m,\u001b[34m210\u001b[39;49;00m,\u001b[34m218\u001b[39;49;00m,\u001b[34m228\u001b[39;49;00m,\u001b[34m236\u001b[39;49;00m,\u001b[34m243\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m251\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m241\u001b[39;49;00m,\u001b[34m238\u001b[39;49;00m,\u001b[34m237\u001b[39;49;00m,\u001b[34m237\u001b[39;49;00m,\u001b[34m239\u001b[39;49;00m,\u001b[34m242\u001b[39;49;00m,\u001b[34m245\u001b[39;49;00m,\u001b[34m247\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m248\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m249\u001b[39;49;00m,\u001b[34m250\u001b[39;49;00m,\u001b[34m252\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m253\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m254\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m,\u001b[34m255\u001b[39;49;00m])\r\n",
      "\r\n",
      "\r\n",
      "\u001b[34mclass\u001b[39;49;00m \u001b[04m\u001b[32mMnistTransformer\u001b[39;49;00m(\u001b[36mobject\u001b[39;49;00m):\r\n",
      "\r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32m__init__\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m):\r\n",
      "        \u001b[34mprint\u001b[39;49;00m(\u001b[33m\"\u001b[39;49;00m\u001b[33minit\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m);\r\n",
      "        \r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32mpreProcessMNIST\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m,X):\r\n",
      "        \u001b[33m'''\u001b[39;49;00m\r\n",
      "\u001b[33m        Convert values assumed to be in 0-1 range to a value in 0-255.\u001b[39;49;00m\r\n",
      "\u001b[33m        The remove the training mean needed by the Caffe2 model.\u001b[39;49;00m\r\n",
      "\u001b[33m        Finally reshape the output to that expected by the model\u001b[39;49;00m\r\n",
      "\u001b[33m        '''\u001b[39;49;00m\r\n",
      "        X = X * \u001b[34m255\u001b[39;49;00m\r\n",
      "        X = \u001b[34m255\u001b[39;49;00m - X\r\n",
      "        X = (X.reshape(\u001b[34m784\u001b[39;49;00m) - MEANS).reshape(\u001b[34m28\u001b[39;49;00m,\u001b[34m28\u001b[39;49;00m,\u001b[34m1\u001b[39;49;00m)\r\n",
      "        X = np.transpose(X, (\u001b[34m2\u001b[39;49;00m, \u001b[34m0\u001b[39;49;00m, \u001b[34m1\u001b[39;49;00m))\r\n",
      "        \u001b[34mreturn\u001b[39;49;00m X\r\n",
      "\r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32mtransform_input\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m,X,names):\r\n",
      "        \u001b[34mreturn\u001b[39;49;00m \u001b[36mself\u001b[39;49;00m.preProcessMNIST(X)\r\n",
      "\r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32mtransform_output\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m,X,names):\r\n",
      "        \u001b[34mreturn\u001b[39;49;00m X.reshape(\u001b[34m1\u001b[39;49;00m,\u001b[34m10\u001b[39;49;00m)\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize MnistTransformer.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---> Installing application source...\n",
      "Build completed successfully\n"
     ]
    }
   ],
   "source": [
    "!s2i build . seldonio/seldon-core-s2i-python3:0.13 mnist-caffe2-transformer:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f2ce59df93344847d070d4218d6bccb5a7d1dab48120967a2e213435a69dce9c\r\n"
     ]
    }
   ],
   "source": [
    "!docker run --name \"mnist-transformer\" -d --rm -p 5000:5000 mnist-caffe2-transformer:0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the Seldon Nvidia Inference Server Proxy Model.\n",
    "\n",
    "** CHANGE THE IP ADDRESS BELOW TO THAT OF YOUR RUNNING NVIDIA SERVER **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3bcca7450fadcbb8d54f90af0b40c32f26b875938b742451a4365ced0c5d3ccf\r\n"
     ]
    }
   ],
   "source": [
    "!docker run --name \"tensorrt-proxy\" -d --rm -p 5001:5001 \\\n",
    "    -e PREDICTIVE_UNIT_SERVICE_PORT=5001 \\\n",
    "    -e PREDICTIVE_UNIT_PARAMETERS='[{\"name\":\"url\",\"type\":\"STRING\",\"value\":\"35.204.115.6:8000\"},{\"name\":\"model_name\",\"type\":\"STRING\",\"value\":\"tensorrt_mnist\"},{\"name\":\"protocol\",\"type\":\"STRING\",\"value\":\"HTTP\"}]' \\\n",
    "    seldonio/nvidia-inference-server-proxy:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "TRANSFORMER_URL=\"localhost:5000\"\n",
    "PREDICTOR_URL=\"localhost:5001\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next few steps we illustrate each step by step process and test that out on our running Docker containers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAC7dJREFUeJzt3V2IXIUZxvHnqdUb9ULJaBeNXRukKEKTMATBUiwS0SLEDxQDKRGC8UJRwYtKEBKQgpSq9aIKsVncgB8V1JoLaRUNWKGE3Yj40dgqstFtQjLBinolJm8v9kTWuDM7O3M+Jn3/P5CZOefszMvgf8/MnMkeR4QA5PODpgcA0AziB5IifiAp4geSIn4gKeIHkiJ+ICniB5IifiCpH9b5YMuWLYvx8fE6HxJIZWZmRkeOHHE/2w4Vv+2rJT0q6RRJf4qIB3ttPz4+runp6WEeEkAP7Xa7720Hftlv+xRJf5R0jaRLJK23fcmg9wegXsO8518j6aOI+Dgivpb0rKR15YwFoGrDxH+epE/n3Z4tln2H7c22p21PdzqdIR4OQJmGiX+hDxW+9++DI2J7RLQjot1qtYZ4OABlGib+WUnL590+X9KB4cYBUJdh4p+SdJHtC22fJukWSbvKGQtA1QY+1BcR39i+U9LfNHeobyIi3i9tMgCVGuo4f0S8LOnlkmYBUCO+3gskRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyQ11Fl6bc9I+lLSUUnfRES7jKFQn1tvvbXn+snJyZ7rH3jggZ7r77///qWOhJoMFX/hlxFxpIT7AVAjXvYDSQ0bf0h6xfZe25vLGAhAPYZ92X95RBywfY6kV21/EBFvzN+g+KWwWZIuuOCCIR8OQFmG2vNHxIHi8rCkFyWtWWCb7RHRjoh2q9Ua5uEAlGjg+G2fbvvM49clXSXpvbIGA1CtYV72nyvpRdvH7+fpiPhrKVMBqNzA8UfEx5J+VuIsGEHFL/euduzY0XP9pk2buq4bGxsbaCaUg0N9QFLEDyRF/EBSxA8kRfxAUsQPJFXGv+pDYvv37++5/vPPP++6jkN9zWLPDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJLXo3+23PSHpWkmHI+LSYtnZkv4saVzSjKSbI+K/1Y2Jqqxatarn+p07dw51/3v37u267uKLLx7qvjGcfvb8T0q6+oRl90l6LSIukvRacRvASWTR+CPiDUmfnbB4naTJ4vqkpOtKngtAxQZ9z39uRByUpOLynPJGAlCHyj/ws73Z9rTt6U6nU/XDAejToPEfsj0mScXl4W4bRsT2iGhHRLvVag34cADKNmj8uyRtLK5vlPRSOeMAqMui8dt+RtI/JP3U9qztTZIelLTW9oeS1ha3AZxEFj3OHxHru6y6suRZ0IC1a9dWev+vv/5613UbNmyo9LHRG9/wA5IifiAp4geSIn4gKeIHkiJ+ICniB5IifiAp4geSIn4gKeIHkiJ+ICniB5IifiAp4geSIn4gKeIHkiJ+ICniB5IifiAp4geSIn4gqUX/dDf+v83OzlZ6/zfccEOl94/BsecHkiJ+ICniB5IifiAp4geSIn4gKeIHklr0OL/tCUnXSjocEZcWy7ZJuk1Sp9hsS0S8XNWQqM6ePXsqvf8VK1ZUev8YXD97/iclXb3A8kciYmXxH+EDJ5lF44+INyR9VsMsAGo0zHv+O22/Y3vC9lmlTQSgFoPG/7ikFZJWSjoo6aFuG9rebHva9nSn0+m2GYCaDRR/RByKiKMRcUzSE5LW9Nh2e0S0I6LdarUGnRNAyQaK3/bYvJvXS3qvnHEA1KWfQ33PSLpC0jLbs5K2SrrC9kpJIWlG0u0VzgigAovGHxHrF1i8o4JZ0IAbb7yx5/qtW7fWNAnqxjf8gKSIH0iK+IGkiB9IiviBpIgfSIo/3Z3c1NRU0yOgIez5gaSIH0iK+IGkiB9IiviBpIgfSIr4gaQ4zp/c7t27mx4BDWHPDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8ktWj8tpfb3m17n+33bd9dLD/b9qu2Pywuz6p+XABl6WfP/42keyPiYkmXSbrD9iWS7pP0WkRcJOm14jaAk8Si8UfEwYh4q7j+paR9ks6TtE7SZLHZpKTrqhoSQPmW9J7f9rikVZL2SDo3Ig5Kc78gJJ1T9nAAqtN3/LbPkPS8pHsi4osl/Nxm29O2pzudziAzAqhAX/HbPlVz4T8VES8Uiw/ZHivWj0k6vNDPRsT2iGhHRLvVapUxM4AS9PNpvyXtkLQvIh6et2qXpI3F9Y2SXip/PABV6edPd18u6deS3rX9drFsi6QHJT1ne5OkTyTdVM2IGGWXXXZZz/XLly+vaRIs1aLxR8Sbktxl9ZXljgOgLnzDD0iK+IGkiB9IiviBpIgfSIr4gaQ4RTeGcvTo0Z7rP/jgg67r2u122eNgCdjzA0kRP5AU8QNJET+QFPEDSRE/kBTxA0lxnB9DmZqa6rn+scce67puYmKi7HGwBOz5gaSIH0iK+IGkiB9IiviBpIgfSIr4gaQ4zp/cXXfd1XP9zp07e67fsGFDz/Xbtm1b6kioCXt+ICniB5IifiAp4geSIn4gKeIHkiJ+IKlFj/PbXi5pp6QfSTomaXtEPGp7m6TbJHWKTbdExMtVDYpqrF69uuf6Y8eO1TQJ6tbPl3y+kXRvRLxl+0xJe22/Wqx7JCJ+X914AKqyaPwRcVDSweL6l7b3STqv6sEAVGtJ7/ltj0taJWlPsehO2+/YnrB9Vpef2Wx72vZ0p9NZaBMADeg7fttnSHpe0j0R8YWkxyWtkLRSc68MHlro5yJie0S0I6LdarVKGBlAGfqK3/apmgv/qYh4QZIi4lBEHI2IY5KekLSmujEBlG3R+G1b0g5J+yLi4XnLx+Ztdr2k98ofD0BV+vm0/3JJv5b0ru23i2VbJK23vVJSSJqRdHslEwKoRD+f9r8pyQus4pg+cBLjG35AUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJEX8QFLEDyRF/EBSxA8kRfxAUsQPJOWIqO/B7I6k/fMWLZN0pLYBlmZUZxvVuSRmG1SZs/04Ivr6e3m1xv+9B7enI6Ld2AA9jOpsozqXxGyDamo2XvYDSRE/kFTT8W9v+PF7GdXZRnUuidkG1chsjb7nB9Ccpvf8ABrSSPy2r7b9L9sf2b6viRm6sT1j+13bb9uebniWCduHbb83b9nZtl+1/WFxueBp0hqabZvt/xTP3du2f9XQbMtt77a9z/b7tu8uljf63PWYq5HnrfaX/bZPkfRvSWslzUqakrQ+Iv5Z6yBd2J6R1I6Ixo8J2/6FpK8k7YyIS4tlv5P0WUQ8WPziPCsifjMis22T9FXTZ24uTigzNv/M0pKuk3SrGnzuesx1sxp43prY86+R9FFEfBwRX0t6VtK6BuYYeRHxhqTPTli8TtJkcX1Sc//z1K7LbCMhIg5GxFvF9S8lHT+zdKPPXY+5GtFE/OdJ+nTe7VmN1im/Q9Irtvfa3tz0MAs4tzht+vHTp5/T8DwnWvTMzXU64czSI/PcDXLG67I1Ef9CZ/8ZpUMOl0fEaknXSLqjeHmL/vR15ua6LHBm6ZEw6Bmvy9ZE/LOSls+7fb6kAw3MsaCIOFBcHpb0okbv7MOHjp8ktbg83PA83xqlMzcvdGZpjcBzN0pnvG4i/ilJF9m+0PZpkm6RtKuBOb7H9unFBzGyfbqkqzR6Zx/eJWljcX2jpJcanOU7RuXMzd3OLK2Gn7tRO+N1I1/yKQ5l/EHSKZImIuK3tQ+xANs/0dzeXpo7ienTTc5m+xlJV2juX30dkrRV0l8kPSfpAkmfSLopImr/4K3LbFdo7qXrt2duPv4eu+bZfi7p75LelXSsWLxFc++vG3vuesy1Xg08b3zDD0iKb/gBSRE/kBTxA0kRP5AU8QNJET+QFPEDSRE/kNT/AHPCTzzFq9UKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff1285767f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 28, 28)\n"
     ]
    }
   ],
   "source": [
    "data = gen_mnist_data(mnist)\n",
    "response = rest_transform_input_request(TRANSFORMER_URL,data)\n",
    "transformed = np.array(response['data']['ndarray'])\n",
    "print(transformed.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[[6.36274109e-08]]\n",
      "\n",
      "  [[9.99982357e-01]]\n",
      "\n",
      "  [[1.35594092e-07]]\n",
      "\n",
      "  [[2.00526298e-08]]\n",
      "\n",
      "  [[1.17960089e-05]]\n",
      "\n",
      "  [[8.07224509e-08]]\n",
      "\n",
      "  [[4.73712625e-08]]\n",
      "\n",
      "  [[4.78241873e-06]]\n",
      "\n",
      "  [[6.21992911e-07]]\n",
      "\n",
      "  [[9.71163061e-08]]]]\n"
     ]
    }
   ],
   "source": [
    "response = rest_predict_request(PREDICTOR_URL,transformed)\n",
    "predictions = np.array(response[\"data\"][\"ndarray\"])\n",
    "print(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': None, 'ndarray': [[6.362741089560586e-08, 0.9999823570251465, 1.3559409239860543e-07, 2.005262977888833e-08, 1.1796008948294912e-05, 8.072245094581376e-08, 4.737126246823209e-08, 4.782418727700133e-06, 6.219929105100164e-07, 9.711630610809152e-08]]}}\n"
     ]
    }
   ],
   "source": [
    "response = rest_transform_output_request(TRANSFORMER_URL,predictions)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mnist-transformer\n",
      "tensorrt-proxy\n"
     ]
    }
   ],
   "source": [
    "!docker rm -f mnist-transformer\n",
    "!docker rm -f tensorrt-proxy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test From GCP Cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up GCP Kubernetes Cluster\n",
    "\n",
    "To run the steps below you will need to:\n",
    "\n",
    " 1. Create a GCP Cluster with a GPU node pool with Nvidia V100 GPUs\n",
    " 2. Enable CUDA on the GPU nodes\n",
    " 3. Add an Image Pull Secret so you can download the Nvidia Inference Server \n",
    " \n",
    "#### Create a GCP Cluster\n",
    " This can be done from the Google console or via the command line as shown below. Change the cluster name and zones as appropriate for your setup.\n",
    "\n",
    "```\n",
    " gcloud container clusters create myinferenceserver --num-nodes=2 --region=europe-west4-a\n",
    " gcloud config set container/cluster myinferenceserver\n",
    " gcloud container node-pools create gpu-pool --num-nodes=1 --machine-type=n1-standard-8 --accelerator type=nvidia-tesla-v100,count=1 --region=europe-west4-a\n",
    " gcloud container clusters get-credentials myinferenceserver\n",
    "```\n",
    "\n",
    "#### Enable CUDA on GPU Nodes\n",
    "\n",
    "To enable the CUDA drivers on your GPU nodes run:\n",
    "\n",
    "```\n",
    "kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/stable/nvidia-driver-installer/cos/daemonset-preloaded.yaml\n",
    "```\n",
    "\n",
    "#### Create Image Pull Secret for the Nvidia Repository\n",
    "\n",
    " * [Sign up to the NVIDIA GPU Cloud and get an API Key](https://ngc.nvidia.com/)\n",
    " * Create a kubernetes secret\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "%env NVIDIA_API_KEY=<your-api-key>\n",
    "%env NVIDIA_CLOUD_EMAIL=<your-email-address>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"gke_seldon-demos_europe-west4-a_myinferenceserver\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "secret/ngc created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create secret docker-registry ngc \\\n",
    "    --docker-server=nvcr.io \\\n",
    "    --docker-username=\\$oauthtoken \\\n",
    "    --docker-password=${NVIDIA_API_KEY} --docker-email=${NVIDIA_CLOUD_EMAIL}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run MNIST Inference Graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"tiller-deploy\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"tiller-deploy\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"gke_seldon-demos_europe-west4-a_myinferenceserver\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Tue Apr 16 19:19:04 2019\n",
      "NAMESPACE: seldon-system\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/Secret\n",
      "NAME                                   TYPE    DATA  AGE\n",
      "seldon-operator-webhook-server-secret  Opaque  0     2s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     2s\n",
      "\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME                                 AGE\n",
      "seldon-operator-manager-rolebinding  2s\n",
      "\n",
      "==> v1/StatefulSet\n",
      "NAME                                DESIRED  CURRENT  AGE\n",
      "seldon-operator-controller-manager  1        1        1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                         READY  STATUS             RESTARTS  AGE\n",
      "seldon-spartakus-volunteer-57647c7679-z9fvw  0/1    ContainerCreating  0         0s\n",
      "seldon-operator-controller-manager-0         0/1    ContainerCreating  0         1s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  2s\n",
      "\n",
      "==> v1/ClusterRole\n",
      "seldon-operator-manager-role  2s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE\n",
      "seldon-operator-controller-manager-service  ClusterIP  10.15.255.125  <none>       443/TCP  2s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        1        1           0          1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        1s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../../helm-charts/seldon-core-operator --name seldon-core --set usageMetrics.enabled=true --set engine.port=7000   --namespace seldon-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "partitioned roll out complete: 1 new pods have been updated...\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-controller-manager -n seldon-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Ingress\n",
    "Please note: There are reported gRPC issues with ambassador (see https://github.com/SeldonIO/seldon-core/issues/473)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   ambassador\n",
      "LAST DEPLOYED: Tue Apr 16 17:35:27 2019\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ServiceAccount\n",
      "NAME        SECRETS  AGE\n",
      "ambassador  1        0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME        AGE\n",
      "ambassador  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME        AGE\n",
      "ambassador  0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME               TYPE          CLUSTER-IP    EXTERNAL-IP  PORT(S)                     AGE\n",
      "ambassador-admins  ClusterIP     10.15.252.67  <none>       8877/TCP                    0s\n",
      "ambassador         LoadBalancer  10.15.251.65  <pending>    80:31240/TCP,443:31402/TCP  0s\n",
      "\n",
      "==> v1/Deployment\n",
      "NAME        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "ambassador  3        3        3           0          0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                         READY  STATUS             RESTARTS  AGE\n",
      "ambassador-7fb5454fb8-bxp7g  0/1    ContainerCreating  0         0s\n",
      "ambassador-7fb5454fb8-flzbj  1/1    Terminating        0         3m\n",
      "ambassador-7fb5454fb8-jqkxk  0/1    ContainerCreating  0         0s\n",
      "ambassador-7fb5454fb8-kfbls  0/1    Terminating        0         3m\n",
      "ambassador-7fb5454fb8-mnxrj  0/1    ContainerCreating  0         0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Congratuations! You've successfully installed Ambassador.\n",
      "\n",
      "For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.\n",
      "\n",
      "To get the IP address of Ambassador, run the following commands:\n",
      "NOTE: It may take a few minutes for the LoadBalancer IP to be available.\n",
      "     You can watch the status of by running 'kubectl get svc -w  --namespace seldon ambassador'\n",
      "\n",
      "  On GKE/Azure:\n",
      "  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n",
      "\n",
      "  On AWS:\n",
      "  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')\n",
      "\n",
      "  echo http://$SERVICE_IP:\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install stable/ambassador --name ambassador --set crds.keep=false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"ambassador\" rollout to finish: 0 of 3 updated replicas are available...\n",
      "Waiting for deployment \"ambassador\" rollout to finish: 1 of 3 updated replicas are available...\n",
      "Waiting for deployment \"ambassador\" rollout to finish: 2 of 3 updated replicas are available...\n",
      "deployment \"ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deployment.apps/ambassador"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   nvidia-mnist\n",
      "LAST DEPLOYED: Tue Apr 16 19:19:38 2019\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME          AGE\n",
      "nvidia-mnist  1s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install nvidia-mnist --namespace seldon --name nvidia-mnist --set tfserving.model_base_path=${MODEL_REPOSITORY_BUCKET}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "!helm template nvidia-mnist --name nvidia-mnist --namespace seldon --set tfserving.model_base_path=${MODEL_REPOSITORY_BUCKET} > mnist.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "!sed '1,2d' mnist.json > tmp.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"208pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 208.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-239 204,-239 204,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-227 192,-227 192,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"100\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- mnist&#45;transformer0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>mnist&#45;transformer0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"158,-196 42,-196 42,-160 158,-160 158,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"100\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">mnist&#45;transformer</text>\n",
       "</g>\n",
       "<!-- mnist&#45;transformer0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>mnist&#45;transformer0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"48\" cy=\"-106\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"48\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- mnist&#45;transformer0&#45;&gt;mnist&#45;transformer0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>mnist&#45;transformer0&#45;&gt;mnist&#45;transformer0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M87.1461,-159.697C80.7907,-151.141 73.0133,-140.672 66.0702,-131.325\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"68.766,-129.085 59.9931,-123.145 63.1468,-133.259 68.766,-129.085\"/>\n",
       "</g>\n",
       "<!-- nvidia&#45;proxy -->\n",
       "<g id=\"node3\" class=\"node\"><title>nvidia&#45;proxy</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"184,-124 98,-124 98,-88 184,-88 184,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"141\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">nvidia&#45;proxy</text>\n",
       "</g>\n",
       "<!-- mnist&#45;transformer0&#45;&gt;nvidia&#45;proxy -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>mnist&#45;transformer0&#45;&gt;nvidia&#45;proxy</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M110.135,-159.697C114.852,-151.644 120.562,-141.894 125.782,-132.982\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"128.948,-134.502 130.982,-124.104 122.908,-130.964 128.948,-134.502\"/>\n",
       "</g>\n",
       "<!-- nvidia&#45;proxyendpoint -->\n",
       "<g id=\"node4\" class=\"node\"><title>nvidia&#45;proxyendpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"141\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"141\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- nvidia&#45;proxy&#45;&gt;nvidia&#45;proxyendpoint -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>nvidia&#45;proxy&#45;&gt;nvidia&#45;proxyendpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M141,-87.6966C141,-79.9827 141,-70.7125 141,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"144.5,-62.1043 141,-52.1043 137.5,-62.1044 144.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fa085a61e10>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"tmp.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mv\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mv\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01my\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"nvidia-mnist\"\u001b[39;49;00m,\r\n",
      "\t\u001b[34;01m\"namespace\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"caffe2-mnist\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mnist-caffe2-transformer:0.1\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mnist-transformer\"\u001b[39;49;00m\r\n",
      "                            },\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/nvidia-inference-server-proxy:0.1\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"nvidia-proxy\"\u001b[39;49;00m\r\n",
      "                            },\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"args\"\u001b[39;49;00m: [\r\n",
      "\t\t\t\t    \u001b[33m\"--model-store=gs://seldon-inference-server-model-store\"\u001b[39;49;00m\r\n",
      "\t\t\t\t],\r\n",
      "\t\t\t\t\u001b[34;01m\"command\"\u001b[39;49;00m: [\r\n",
      "\t\t\t\t    \u001b[33m\"inference_server\"\u001b[39;49;00m\r\n",
      "\t\t\t\t],\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"nvcr.io/nvidia/inferenceserver:18.08.1-py2\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"livenessProbe\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"failureThreshold\"\u001b[39;49;00m: \u001b[34m3\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"handler\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\t\u001b[34;01m\"httpGet\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"path\"\u001b[39;49;00m: \u001b[33m\"/api/health/live\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"port\"\u001b[39;49;00m: \u001b[34m8000\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"scheme\"\u001b[39;49;00m: \u001b[33m\"HTTP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t\t}\r\n",
      "\t\t\t\t    },\r\n",
      "\t\t\t\t    \u001b[34;01m\"initialDelaySeconds\"\u001b[39;49;00m: \u001b[34m5\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"periodSeconds\"\u001b[39;49;00m: \u001b[34m5\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"successThreshold\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"timeoutSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "\t\t\t\t},\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"inference-server\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"ports\"\u001b[39;49;00m: [\r\n",
      "\t\t\t\t    {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"containerPort\"\u001b[39;49;00m: \u001b[34m8000\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t\u001b[34;01m\"protocol\"\u001b[39;49;00m: \u001b[33m\"TCP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    },\r\n",
      "\t\t\t\t    {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"containerPort\"\u001b[39;49;00m: \u001b[34m8001\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t\u001b[34;01m\"protocol\"\u001b[39;49;00m: \u001b[33m\"TCP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    },\r\n",
      "\t\t\t\t    {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"containerPort\"\u001b[39;49;00m: \u001b[34m8002\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t\u001b[34;01m\"protocol\"\u001b[39;49;00m: \u001b[33m\"TCP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t],\r\n",
      "\t\t\t\t\u001b[34;01m\"readinessProbe\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"failureThreshold\"\u001b[39;49;00m: \u001b[34m3\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"handler\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\t\u001b[34;01m\"httpGet\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"path\"\u001b[39;49;00m: \u001b[33m\"/api/health/ready\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"port\"\u001b[39;49;00m: \u001b[34m8000\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t    \u001b[34;01m\"scheme\"\u001b[39;49;00m: \u001b[33m\"HTTP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t\t}\r\n",
      "\t\t\t\t    },\r\n",
      "\t\t\t\t    \u001b[34;01m\"initialDelaySeconds\"\u001b[39;49;00m: \u001b[34m5\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"periodSeconds\"\u001b[39;49;00m: \u001b[34m5\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"successThreshold\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"timeoutSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "\t\t\t\t},\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"limits\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"nvidia.com/gpu\"\u001b[39;49;00m: \u001b[33m\"1\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    },\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"cpu\"\u001b[39;49;00m: \u001b[33m\"100m\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\t\u001b[34;01m\"nvidia.com/gpu\"\u001b[39;49;00m: \u001b[33m\"1\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t},\r\n",
      "\t\t\t\t\u001b[34;01m\"securityContext\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"runAsUser\"\u001b[39;49;00m: \u001b[34m1000\u001b[39;49;00m\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\t\u001b[34;01m\"imagePullSecrets\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"ngc\"\u001b[39;49;00m\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t]\r\n",
      "\t\t    }\r\n",
      "\t\t}],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mnist-transformer\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: { \u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"TRANSFORMER\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"nvidia-proxy\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: { \u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "\t\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m:\r\n",
      "\t\t\t    [\r\n",
      "\t\t\t\t{\r\n",
      "\t\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m:\u001b[33m\"url\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"STRING\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m:\u001b[33m\"127.0.0.1:8000\"\u001b[39;49;00m\r\n",
      "\t\t\t\t},\r\n",
      "\t\t\t\t{\r\n",
      "\t\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m:\u001b[33m\"model_name\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"STRING\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m:\u001b[33m\"tensorrt_mnist\"\u001b[39;49;00m\r\n",
      "\t\t\t\t},\r\n",
      "\t\t\t\t{\r\n",
      "\t\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m:\u001b[33m\"protocol\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"STRING\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m:\u001b[33m\"HTTP\"\u001b[39;49;00m\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    ]\r\n",
      "\t\t\t}\r\n",
      "\t\t    ]\r\n",
      "                },\r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mnist-nvidia\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize mnist.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Port forward Ambassador**\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADJBJREFUeJzt3WGoXPWZx/Hfz5ggmiJKJjYYs7dbRVYCpssYF1wWl5Jil2osUmnAclfKpmIFA31R8U0FWZBl29oXSyQ1MSm0toXUNYruVmXFFkpxlNikZncr5W57Tbh3ooWkINTosy/uSbkmd87MnTlnziTP9wNhZs5z5pyHQ373nJn/zPwdEQKQzwVNNwCgGYQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSF45zZ2vWrImpqalx7hJIZWZmRsePH/cg644Uftu3SPqOpBWSHo+IR8rWn5qaUqfTGWWXAEq02+2B1x36st/2Ckn/Jumzkq6TtM32dcNuD8B4jfKaf7OktyLitxHxJ0k/lLS1mrYA1G2U8F8p6feLHs8Wyz7C9nbbHdudbrc7wu4AVGmU8C/1psJZ3w+OiF0R0Y6IdqvVGmF3AKo0SvhnJV216PF6SUdHawfAuIwS/lclXWP7E7ZXSfqipAPVtAWgbkMP9UXEKdv3SfpPLQz17YmIX1fWGYBajTTOHxHPSXquol4AjBEf7wWSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gqbFO0Y3xO3ToUGn9+uuvH2n7EWdN0vQRdu/Zou+6667S5z7++OOl9VWrVpXWUY4zP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k5X7jtKVPtmcknZT0gaRTEdEuW7/dbken0xl6f1i+EydOlNb37t1bWn/22WdL6y+++GJpvWycv5+TJ0+W1i+++OKht32+arfb6nQ6Ax30Kj7k8/cRcbyC7QAYIy77gaRGDX9I+qnt12xvr6IhAOMx6mX/TRFx1PZaSS/Y/u+IeGXxCsUfhe2StGHDhhF3B6AqI535I+JocTsv6SlJm5dYZ1dEtCOi3Wq1RtkdgAoNHX7bl9j+2On7kj4j6XBVjQGo1yiX/VdIeqoYyrlQ0g8i4j8q6QpA7UYa518uxvnPPxdcUH7xyDj/eC1nnJ+hPiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFJM0Y1Szz//fG3bXrt2bWm939eFMRqOLpAU4QeSIvxAUoQfSIrwA0kRfiApwg8kxTg/Sj388MO1bfu2224rrV900UW17Ruc+YG0CD+QFOEHkiL8QFKEH0iK8ANJEX4gqb7j/Lb3SPqcpPmI2Fgsu1zSjyRNSZqRdGdE/KG+NtGUubm52ra9cuXK2raN/gY58++VdMsZyx6Q9FJEXCPppeIxgHNI3/BHxCuS3j1j8VZJ+4r7+yTdXnFfAGo27Gv+KyLimCQVt+W/xwRg4tT+hp/t7bY7tjvdbrfu3QEY0LDhn7O9TpKK2/leK0bErohoR0S71WoNuTsAVRs2/AckTRf3pyU9XU07AMalb/htPynpF5KutT1r+8uSHpG0xfZvJG0pHgM4h/Qd54+IbT1Kn664FzTgxIkTpfX3339/pO2vXr26Z23Hjh0jbRuj4RN+QFKEH0iK8ANJEX4gKcIPJEX4gaT46e7kXn755dL622+/PdL2b7zxxp61q6++eqRtYzSc+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcb5kzt48GDTLaAhnPmBpAg/kBThB5Ii/EBShB9IivADSRF+ICnG+c9zR48eLa3v3r27tB4RtdbRHM78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5BU33F+23skfU7SfERsLJY9JOmfJHWL1R6MiOfqahLDe+edd0rrs7OzpXXbI+3/1ltvHen5qM8gZ/69km5ZYvm3I2JT8Y/gA+eYvuGPiFckvTuGXgCM0Siv+e+z/Svbe2xfVllHAMZi2PDvlPRJSZskHZP0zV4r2t5uu2O70+12e60GYMyGCn9EzEXEBxHxoaTvStpcsu6uiGhHRLvVag3bJ4CKDRV+2+sWPfy8pMPVtANgXAYZ6ntS0s2S1tielfQNSTfb3iQpJM1I+kqNPQKoQd/wR8S2JRaXfwkcaaxfv760fvfdd4+pEywXn/ADkiL8QFKEH0iK8ANJEX4gKcIPJMVPd5/ndu7cWev277///tL6ihUrat0/hseZH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSYpz/PHfq1Klat//mm2+W1u+9996etb1791bcDZaDMz+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJMU4/3ngvffe61l75plnat33E088UVq/9NJLe9b2799f+tw77rhjqJ4wGM78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5BU33F+21dJ+p6kj0v6UNKuiPiO7csl/UjSlKQZSXdGxB/qaxW9PPbYYz1r8/PzY+zkbO12u2dt69atY+wEZxrkzH9K0tci4q8k/Y2kr9q+TtIDkl6KiGskvVQ8BnCO6Bv+iDgWEa8X909KOiLpSklbJe0rVtsn6fa6mgRQvWW95rc9JelTkn4p6YqIOCYt/IGQtLbq5gDUZ+Dw214tab+kHRFxYhnP2267Y7vT7XaH6RFADQYKv+2VWgj+9yPiJ8XiOdvrivo6SUu+sxQRuyKiHRHtVqtVRc8AKtA3/LYtabekIxHxrUWlA5Kmi/vTkp6uvj0AdRnkK703SfqSpEO2DxbLHpT0iKQf2/6ypN9J+kI9LaKfsq/01u3aa68tre/evbtn7cIL+UZ5k/oe/Yj4uST3KH+62nYAjAuf8AOSIvxAUoQfSIrwA0kRfiApwg8kxUArRnLPPfeU1jds2DCmTrBcnPmBpAg/kBThB5Ii/EBShB9IivADSRF+ICnG+c8DN9xwQ8/axo0bS597+PDh0vqjjz5aWp+eni6tY3Jx5geSIvxAUoQfSIrwA0kRfiApwg8kRfiBpBjnPw9s2bKlZ+2NN94YYyc4l3DmB5Ii/EBShB9IivADSRF+ICnCDyRF+IGk+obf9lW2/8v2Edu/tn1/sfwh22/bPlj8+4f62wVQlUE+5HNK0tci4nXbH5P0mu0Xitq3I+Jf62sPQF36hj8ijkk6Vtw/afuIpCvrbgxAvZb1mt/2lKRPSfplseg+27+yvcf2ZT2es912x3an2+2O1CyA6gwcfturJe2XtCMiTkjaKemTkjZp4crgm0s9LyJ2RUQ7ItqtVquClgFUYaDw216pheB/PyJ+IkkRMRcRH0TEh5K+K2lzfW0CqNog7/Zb0m5JRyLiW4uWr1u02ucllf8MLICJMsi7/TdJ+pKkQ7YPFsselLTN9iZJIWlG0ldq6RBALQZ5t//nkrxE6bnq2wEwLnzCD0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kJQjYnw7s7uS/m/RojWSjo+tgeWZ1N4mtS+J3oZVZW9/ERED/V7eWMN/1s7tTkS0G2ugxKT2Nql9SfQ2rKZ647IfSIrwA0k1Hf5dDe+/zKT2Nql9SfQ2rEZ6a/Q1P4DmNH3mB9CQRsJv+xbb/2P7LdsPNNFDL7ZnbB8qZh7uNNzLHtvztg8vWna57Rds/6a4XXKatIZ6m4iZm0tmlm702E3ajNdjv+y3vULS/0raImlW0quStkXEm2NtpAfbM5LaEdH4mLDtv5P0R0nfi4iNxbJ/kfRuRDxS/OG8LCK+PiG9PSTpj03P3FxMKLNu8czSkm6X9I9q8NiV9HWnGjhuTZz5N0t6KyJ+GxF/kvRDSVsb6GPiRcQrkt49Y/FWSfuK+/u08J9n7Hr0NhEi4lhEvF7cPynp9MzSjR67kr4a0UT4r5T0+0WPZzVZU36HpJ/afs329qabWcIVxbTpp6dPX9twP2fqO3PzOJ0xs/TEHLthZryuWhPhX2r2n0kacrgpIv5a0mclfbW4vMVgBpq5eVyWmFl6Igw743XVmgj/rKSrFj1eL+loA30sKSKOFrfzkp7S5M0+PHd6ktTidr7hfv5skmZuXmpmaU3AsZukGa+bCP+rkq6x/QnbqyR9UdKBBvo4i+1LijdiZPsSSZ/R5M0+fEDSdHF/WtLTDfbyEZMyc3OvmaXV8LGbtBmvG/mQTzGU8aikFZL2RMQ/j72JJdj+Sy2c7aWFSUx/0GRvtp+UdLMWvvU1J+kbkv5d0o8lbZD0O0lfiIixv/HWo7ebtXDp+ueZm0+/xh5zb38r6WeSDkn6sFj8oBZeXzd27Er62qYGjhuf8AOS4hN+QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeS+n+3kZSmVwbCVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa082b84748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"pvms7jocoktb37gkb8hpj4cej0\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"mnist-transformer\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"nvidia-proxy\": \"seldonio/nvidia-inference-server-proxy:0.1\",\n",
      "      \"mnist-transformer\": \"seldonio/mnist-caffe2-transformer:0.1\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"t:0\", \"t:1\", \"t:2\", \"t:3\", \"t:4\", \"t:5\", \"t:6\", \"t:7\", \"t:8\", \"t:9\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 10, 1, 1],\n",
      "      \"values\": [2.5883252874336904E-7, 0.9981641173362732, 2.274509461130947E-5, 8.619469736004248E-5, 3.228335117455572E-6, 2.4672864640251646E-8, 4.060555909290997E-7, 0.0016722471918910742, 5.0836355512728915E-5, 1.2317170394737786E-8]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "data = gen_mnist_data(mnist)\n",
    "data = data.reshape((784))\n",
    "rest_request_ambassador(\"nvidia-mnist\",endpoint=\"localhost:8003\",arr=data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analytics and Load Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-analytics\n",
      "LAST DEPLOYED: Mon Sep 24 14:40:46 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME        AGE\n",
      "prometheus  0s\n",
      "\n",
      "==> v1/Job\n",
      "NAME                            DESIRED  SUCCESSFUL  AGE\n",
      "grafana-prom-import-dashboards  1        0           0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                      TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)       AGE\n",
      "alertmanager              ClusterIP  10.39.243.208  <none>       80/TCP        0s\n",
      "grafana-prom              NodePort   10.39.246.121  <none>       80:30588/TCP  0s\n",
      "prometheus-node-exporter  ClusterIP  None           <none>       9100/TCP      0s\n",
      "prometheus-seldon         ClusterIP  10.39.253.10   <none>       80/TCP        0s\n",
      "\n",
      "==> v1beta1/DaemonSet\n",
      "NAME                      DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR  AGE\n",
      "prometheus-node-exporter  1        1        0      1           0          <none>         0s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                       DATA  AGE\n",
      "alertmanager-server-conf   1     0s\n",
      "grafana-import-dashboards  7     0s\n",
      "prometheus-rules           4     0s\n",
      "prometheus-server-conf     1     0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME        AGE\n",
      "prometheus  0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "alertmanager-deployment  1        1        1           0          0s\n",
      "grafana-prom-deployment  1        1        1           0          0s\n",
      "prometheus-deployment    1        1        1           0          0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                      READY  STATUS             RESTARTS  AGE\n",
      "grafana-prom-import-dashboards-wrtnb      0/1    ContainerCreating  0         0s\n",
      "alertmanager-deployment-557b99ccf8-ztdnk  0/1    ContainerCreating  0         0s\n",
      "grafana-prom-deployment-dd84b7788-zwwls   0/1    ContainerCreating  0         0s\n",
      "prometheus-node-exporter-zg9cg            0/1    ContainerCreating  0         0s\n",
      "prometheus-deployment-78dd89b44f-8ntcw    0/1    Pending            0         0s\n",
      "\n",
      "==> v1/Secret\n",
      "NAME                 TYPE    DATA  AGE\n",
      "grafana-prom-secret  Opaque  1     0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME        SECRETS  AGE\n",
      "prometheus  1        0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../../helm-charts/seldon-core-analytics --name seldon-core-analytics \\\n",
    "    --set grafana_prom_admin_password=password \\\n",
    "    --set persistence.enabled=false \\\n",
    "    --namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "node \"gke-cluster-1-default-pool-5e22f4e3-djd7\" labeled\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl label nodes $(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') role=locust"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   loadtest\n",
      "LAST DEPLOYED: Mon Sep 24 14:40:53 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ReplicationController\n",
      "NAME             DESIRED  CURRENT  READY  AGE\n",
      "locust-slave-1   1        1        0      0s\n",
      "locust-master-1  1        1        0      0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME             TYPE      CLUSTER-IP    EXTERNAL-IP  PORT(S)                                       AGE\n",
      "locust-master-1  NodePort  10.39.254.17  <none>       5557:31880/TCP,5558:30185/TCP,8089:32570/TCP  0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                   READY  STATUS             RESTARTS  AGE\n",
      "locust-slave-1-p5n6f   0/1    ContainerCreating  0         0s\n",
      "locust-master-1-ksk7g  0/1    ContainerCreating  0         0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install seldon-core-loadtesting --name loadtest  \\\n",
    "    --namespace seldon \\\n",
    "    --repo https://storage.googleapis.com/seldon-charts \\\n",
    "    --set locust.script=mnist_rest_locust.py \\\n",
    "    --set locust.host=http://caffe2-mnist:8000 \\\n",
    "    --set oauth.enabled=false \\\n",
    "    --set locust.hatchRate=1 \\\n",
    "    --set locust.clients=1 \\\n",
    "    --set loadtest.sendFeedback=1 \\\n",
    "    --set locust.minWait=0 \\\n",
    "    --set locust.maxWait=0 \\\n",
    "    --set replicaCount=1 \\\n",
    "    --set data.size=784\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should port-foward the grafana dashboard\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app=grafana-prom-server -o jsonpath='{.items[0].metadata.name}') -n seldon 3000:3000\n",
    "```\n",
    "\n",
    "You can then view an analytics dashboard inside the cluster at http://localhost:3000/dashboard/db/prediction-analytics?refresh=5s&orgId=1. Your IP address may be different. get it via minikube ip. Login with:\n",
    "\n",
    "    Username : admin\n",
    "\n",
    "    password : password (as set when starting seldon-core-analytics above)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
